Multicenter Diagnostic Evaluation of a Novel Coronavirus Antigen Lateral Flow Test among Symptomatic Individuals in Brazil and the United Kingdom

ABSTRACT The COVID-19 pandemic has led to the commercialization of many antigen-based rapid diagnostic tests (Ag-RDTs), requiring independent evaluations. This report describes the clinical evaluation of the Novel Coronavirus 2019-nCoV Antigen Test (Colloidal Gold) (Beijing Hotgen Biotech Co., Ltd.), at two sites within Brazil and one in the United Kingdom. The collected samples (446 nasal swabs from Brazil and 246 nasopharyngeal samples from the UK) were analyzed by the Ag-RDT and compared to reverse transcription-quantitative PCR (RT-qPCR). Analytical evaluation of the Ag-RDT was performed using direct culture supernatants of SARS-CoV-2 strains from the wild-type (B.1), Alpha (B.1.1.7), Delta (B.1.617.2), Gamma (P.1), and Omicron (B.1.1.529) lineages. An overall sensitivity and specificity of 88.2% (95% confidence interval [CI], 81.3 to 93.3) and 100.0% (95% CI, 99.1 to 100.0), respectively, were obtained for the Brazilian and UK cohorts. The analytical limit of detection was determined as 1.0 × 103 PFU/mL (Alpha), 2.5 × 102 PFU/mL (Delta), 2.5 × 103 PFU/mL (Gamma), and 1.0 × 103 PFU/mL (Omicron), giving a viral copy equivalent of approximately 2.1 × 104 copies/mL, 9.0 × 105 copies/mL, 1.7 × 106 copies/mL, and 1.8 × 105 copies/mL for the Ag-RDT, respectively. Overall, while a higher sensitivity was claimed by the manufacturers than that found in this study, this evaluation finds that the Ag-RDT meets the WHO minimum performance requirements for sensitivity and specificity of COVID-19 Ag-RDTs. This study illustrates the comparative performance of the Hotgen Ag-RDT across two global settings and considers the different approaches in evaluation methods. IMPORTANCE Since the beginning of the SARS-CoV-2 pandemic, we have witnessed growing numbers of antigen rapid diagnostic tests (Ag-RDTs) being brought to market. In the United Kingdom, this was somewhat controlled indirectly as the government offered free tests from a small number of companies. However, as this has now ceased, individuals are responsible for their own acquisition of test kits. Similarly in Brazil, as of January 2022, pharmacies and other health care retailers are permitted to sell Ag-RDTs directly to the community. Many of these Ag-RDTs have not been externally evaluated, and results are not readily available to the public. Thus, there is now a need for a transparent evaluation of Ag-RDTs with both analytical and clinical evaluation. We present an independent review of the Novel Coronavirus 2019-nCoV Antigen Test (Colloidal Gold) (Beijing Hotgen Biotech Co., Ltd.), at two sites within Brazil and one in the United Kingdom.


Reviewer #2 (Comments for the Author):
Excellent study comparing the Hotgen Ag-RDT performance in different contexts.
My point: -"The LOD using Omicron lineage was comparable to the other lineages, suggesting that sensitivity will not be affected by this new VOC." The omicron variant may have N-or S-gene mutations which may affect the sensitivity of antigen tests. Due to that, some tests had to be modified to detect omicron. Could you discuss about the most frequent mutations of the omicron variant and how they can affect the sensitivity of this specific antigen test? https://www.fda.gov/medical-devices/coronavirus-covid-19-and-medical-devices/sars-cov-2-viral-mutations-impact-covid-19-tests Staff Comments:

Preparing Revision Guidelines
To submit your modified manuscript, log onto the eJP submission site at https://spectrum.msubmit.net/cgi-bin/main.plex. Go to Author Tasks and click the appropriate manuscript title to begin the revision process. The information that you entered when you first submitted the paper will be displayed. Please update the information as necessary. Here are a few examples of required updates that authors must address: • Point-by-point responses to the issues raised by the reviewers in a file named "Response to Reviewers," NOT IN YOUR COVER LETTER. • Upload a compare copy of the manuscript (without figures) as a "Marked-Up Manuscript" file. • Each figure must be uploaded as a separate file, and any multipanel figures must be assembled into one file. For complete guidelines on revision requirements, please see the journal Submission and Review Process requirements at https://journals.asm.org/journal/Spectrum/submission-review-process. Submissions of a paper that does not conform to Microbiology Spectrum guidelines will delay acceptance of your manuscript. " Please return the manuscript within 60 days; if you cannot complete the modification within this time period, please contact me. If you do not wish to modify the manuscript and prefer to submit it to another journal, please notify me of your decision immediately so that the manuscript may be formally withdrawn from consideration by Microbiology Spectrum.
If your manuscript is accepted for publication, you will be contacted separately about payment when the proofs are issued; please follow the instructions in that e-mail. Arrangements for payment must be made before your article is published. For a complete list of Publication Fees, including supplemental material costs, please visit our website.
Corresponding authors may join or renew ASM membership to obtain discounts on publication fees. Need to upgrade your membership level? Please contact Customer Service at Service@asmusa.org.
Thank you for submitting your paper to Microbiology Spectrum. Thank you for allowing us to submit a revision of our manuscript entitled "Multicentre diagnostic evaluation of a novel coronavirus antigen lateral flow test among symptomatic individuals in Brazil and The United Kingdom". We appreciate the careful review and constructive suggestions. We thank the referees for their very helpful comments that have improved our manuscript.

Rachel Louise Byrne
Please find below our response to all comments. For clarity, reviewer comments are in black and ours are consistently in red. The revision has been developed in consultation with all coauthors, and each author has given approval to the final form of this revision.
Thank you for your consideration, please don't hesitate to contact me if you require any clarification on the edits made. Sincerely,

Is there any clinical implication of a fall in sensitivity with a rising Ct value? what are the author's interpretations?
A very good question. We are not clinicians so have not commented on the clinical implications for this study and focused rather on at home or self-testing. A rising Ct value would indicate a low viral load thus a fall in sensitivity would be expected but also necessary. A very sensitive test past the level of clinical infection (individuals capable of transmitting the virus) could result in individuals missing workdays/quarantining when it's not necessary.
3. Any recommendations that can be proposed by authors for future trials for Antigen kit testing We feel this was addressed in our introduction where we state the need for clinical evaluation in an independent setting. "There is a myriad of Ag-RDTs in the market, however their performance varies between brands [5]. The WHO published interim guidance in October 2021 highlighting that despite hundreds of Ag-RDTs test brands available on the market, the number of those examined in published reports is still relatively limited [6]. The estimates on performance should also be cautiously interpreted in the context of their methodological limitations and the settings in which they were conducted." Reviewer #2 (Comments for the Author): Excellent study comparing the Hotgen Ag-RDT performance in different contexts.
My point: -"The LOD using Omicron lineage was comparable to the other lineages, suggesting that sensitivity will not be affected by this new VOC." The omicron variant may have N-or S-gene mutations which may affect the sensitivity of antigen tests. Due to that, some tests had to be modified to detect omicron. Could you discuss about the most frequent mutations of the omicron variant and how they can affect the sensitivity of this specific antigen test? https://www.fda.gov/medical-devices/coronavirus-covid-19-and-medical-devices/sars-cov-2-viralmutations-impact-covid-19-tests Whilst reviewer two makes an excellent point we feel this falls outside the remit of this paper. We acknowledge other manufacturers have had to make adaptations, but our evaluation is only of Hotgen. We included all lineages to evaluate its use in variants of concerns to which we found no statistical difference.
We would support a review paper discussing these limitations and offering considerations to evaluation sites to conduct robust studies. Your manuscript has been accepted, and I am forwarding it to the ASM Journals Department for publication. You will be notified when your proofs are ready to be viewed.
The ASM Journals program strives for constant improvement in our submission and publication process. Please tell us how we can improve your experience by taking this quick Author Survey.
As an open-access publication, Spectrum receives no financial support from paid subscriptions and depends on authors' prompt payment of publication fees as soon as their articles are accepted. You will be contacted separately about payment when the proofs are issued; please follow the instructions in that e-mail. Arrangements for payment must be made before your article is published. For a complete list of Publication Fees, including supplemental material costs, please visit our website.
ASM policy requires that data be available to the public upon online posting of the article, so please verify all links to sequence records, if present, and make sure that each number retrieves the full record of the data. If a new accession number is not linked or a link is broken, provide production staff with the correct URL for the record. If the accession numbers for new data are not publicly accessible before the expected online posting of the article, publication of your article may be delayed; please contact the ASM production staff immediately with the expected release date.